Evaluation of scientific quality of YouTube video content related to orthodontic pain management

ABSTRACT Introduction: With the abundant use of the internet, patients undergoing or interested in orthodontic treatment try to use it to obtain information on pain during treatment. However, YouTube™ is unregulated and may potentially contain inaccurate information. Objectives: Thus, this study aimed to evaluate the scientific quality of the videos on YouTube™ related to orthodontic pain management. Methods: A total of 62 videos related to orthodontic pain management were included in the study. All videos were evaluated by two experienced orthodontists. The video uploader, content, length, upload date, time since upload, number of views, comments, likes, dislikes, Interaction index, and Viewing rate of the videos were recorded and evaluated. The videos were scored using the Quality Criteria for Consumer Health Information (DISCERN), Global Quality Scale (GQS), and Audio-Visual Quality (AVQ), and divided into two groups: Doctors and Non-doctors. Results: The mean DISCERN score was 2.56 ± 0.91, the GQS score was 2.56 ± 1.06, and AVQ was 2.48 ± 0.68. A statistically significant difference was found in DISCERN score of videos uploaded by Doctors compared to Non-doctors, but no statistically significant difference was found in GQS and AVQ scores between both groups (p> 0.05). Conclusions: The videos uploaded by Doctors were better in terms of quality and reliability, as compared to Non-doctors; and the AVQ of the videos uploaded by both groups was adequate. Despite that, both groups did not serve as a good source of information. YouTube™ cannot be considered a reliable source of information in terms of quality and reliability on videos related to orthodontic pain management.


INTRODUCTION
Traditionally, the main responsibility for health information lay with health professionals and organizations.However, with the advancement of science and technology, the use of the internet has become fundamental in our day-to-day activities.A significant population of the world has access to the worldwide web, and multiple websites provide information about modern healthcare modalities.Moreover, with the rise of social media platforms, information can now be disseminated to a wider range of people.Recent surveys report that 80% of internet users access online health information. 1Patients are able to access health information at a reduced cost, compared to a professional healthcare consultation.Patients not only seek medical information but also scroll for diagnosis and treatment purposes. 2The online media content feed is full of information and misinformation that allows patients to cross-reference information provided by the doctor, as well as gather new, additional or conflicting material. 3uTube™ (www.youtube.com) is a pioneer video-sharing platform and is the second-largest search engine, next to Google.
YouTube™ is available in 80 languages, reaching 95% of the Internet population.Most YouTube™ users fall in the age group of 15-35 years. 4Patients prefer YouTube™ because of its ease of access and social nature, providing both audio and visual media that gives a vivid understanding of the subject.Its popularity has made it a powerful tool for influencing individual decisions and promoting their well-being. 5,6in is one of the most negative effects of orthodontic treatment, and is of a major concern to parents, patients and clinicians. 7,8Orthodontic pain is due to changes in the blood flow following force application, leading to cascades of inflammatory reactions, releasing various chemical and neurogenic mediators that express a hyperalgesic response. 9,10The incidence of pain symptoms among orthodontically treated patients has been reported to be around 70% to 95%.Researchers have shown that fear of anticipated pain keeps some patients away from orthodontic treatment.One in ten orthodontic patients may discontinue treatment due to pain disorders during the early stages of treatment. 11Given the growing influence of the internet, it can be assumed that patients undergoing or interested in orthodontic treatment will attempt to use it to obtain information on pain during treatment.As YouTube™ is unregulated and is firmly built upon the principle of free expression, it may contain potentially inaccurate information, particularly because of anecdotal reporting and personal opinion. 12Thus, this study aimed to evaluate the content quality, reliability, and audio-visual quality (AVQ) of the videos related to orthodontic pain management available on YouTube.

YOUTUBE™ VIDEOS SEARCH AND SELECTION
This study did not require ethics committee approval as it used publicly available data.A search was made on YouTube™ in January 2023 for videos related to orthodontic pain management, using two search terms: "orthodontic pain management" and "braces pain management".To avoid any bias, the computer history and cookies were deleted.The videos were sorted using the ''relevance'' filter, a default option on YouTube, which uses a complex algorithm based on the number of views, upload date, rating, comments, etc. Ads displayed by YouTube™ at the beginning and end of the search results were not taken into account.
Previous research has indicated that 95% of users who perform an online search on YouTube™ do not watch more than the first 60 videos in the result, and most studies utilizing YouTube™ as a search engine have used 60-200 videos. 13,14In the present study, the first 250 videos for each search term (i.e., a total of 500 videos) excluding YouTube™ shorts were analyzed.

ASSESSMENT OF VIDEOS
A total of 62 videos were found to be relevant for assessment  The viewers interaction was calculated using the interaction index and viewing rate formulas.Video interaction was calculated through the difference in the total number of "likes" and "dislikes", divided by the total number of views.The AVQ of the videos was assessed according to Sorensen et al. 15 Videos that had clear images and text, good quality graphics or effects, and had no difficulty in understanding spoken words and music were rated as 'good'; while homemade videos, videos with regular quality and average text clarity, sentences that were difficult to understand, distracting audio or background sounds were rated as 'moderate'; and videos containing blurry, grainy, or difficult to understand images and no audio were rated as 'poor'.To evaluate video quality, a 5-point scale, the Global Quality Scale (GQS),was used. 16,17The reliability of information was assessed according to a scoring system derived from the DISCERN tool, based on 5 questions. 18For each question, the answer ''no'' was scored as 0 points and the answer ''yes'' was scored as 1 point.A Reliability Score (RS) was obtained by summing the total of these points.A higher score meant that the video had reliable content. 19(Fig 2 ).

GQS Score DISCERN
Poor quality, poor flow of the site, most information missing, not at all useful for patients Are the aims clear? 1  In this study, a total of 1 to 2 points on both DISCERN and GQS scales was rated as low-quality and misleading content; 3 points, as content of moderate quality; and 4 to 5 points, as useful and good quality content.Twenty videos were randomly selected using an online software (Research Randomizer, https://www.randomizer.org/),and examined again by both operators after two weeks, to calculate the intra-rater reliability.

UPLOADER
The total number of views on the videos was 5,344,065, with a higher viewing count on the Non-doctors videos

ASSESSMENT OF VIDEOS FOR AVQ, GQS, AND RS
The quality of AVQ, GQS and RS was expressed in numbers and percentages.The differences regarding AVQ (p = 0.23) and GQS (p = 0.347) between the Doctors and Non-doctors videos were not significant.The RS of videos was significantly different between the groups: the Doctors' videos had more reliability than the Non-doctors' videos (p=0.049)(Table 2
There was a good correlation between GQS and RS (r = 0.879, p < 0.01).There was a poor correlation among GQS, RS, and AVQ with the number of views, likes, dislikes, and comments on the videos (Table 3).Only 24 (38.7%)videos were uploaded by Doctors, and those had a lesser viewing rate than 38 (61.3%)Non-doctors videos.
These results were similar to the studies examining YouTube™ videos, which reported that videos uploaded by non-doctors were viewed more. 22,23The length of the videos uploaded by the Doctors group was shorter than the Non-doctors group, as they focused more on the topic; whereas the videos in the Non-doctors group were in the weblog format.21 (33.8%)videos were uploaded by the patients, in which they shared their individual experiences, which was consistent with similar studies. 11,22,23 process, and were not asked for the source of information of their videos. 13,19,29The audio and video quality of the image was found to be substantially good, which was similar to the study by Sorenson et al. 15 In the current study, a strong positive correlation was found between content quality and reliability, which was in contrast to the study by Ustdal et al 30 15 in which the number of likes on the videos was correlated with the quality of the video images.

LIMITATIONS
Limitations in this study include the fact that videos were assessed for whether or not they "mentioned" a specific treatment or aetiology, not whether they advocated it.This is a major distinction, as somebody may mention a given treatment or aetiology as wrong, but the data collected does not make that distinction.In addition, even though the most commonly used keywords were searched pertaining to orthodontic pain using the Google Trends application, it should be kept in mind that different videos may be accessed using different keywords.This study generalizes Dental Press J Orthod.2023;28(5):e232386 orthodontic pain, which may be caused due to several reasons, such as mini-implant insertion, wire tightening, etc.Also, this study was conducted only on YouTube™ videos, and other social media sites were not evaluated.So, further study can be conducted evaluating other social media platforms.

CONCLUSION
» According to the study, YouTube™ cannot be considered a reliable source of information in terms of quality and reliability on videos related to orthodontic pain management.
» Even though the videos uploaded by Doctors were better in terms of quality and reliability, as compared to Non-doctors videos; overall, both groups do not serve as a good source of information.
» The audio-visual quality of the uploaded videos by both Doctors and Non-doctors was adequate.
» The viewing rate was better in the Non-doctors videos, but the interaction index of the Doctors videos was better.
The videos uploaded by doctors, hospital channels, or health professionals should also be censored through an effective control mechanism.Collaboration amongst the orthodontic specialists, health professionals, patients, hospital channels, and experts in digital technology is of utmost necessity to ensure that videos related to orthodontic pain management contain relevant, high-quality, and evidence-based information.

Singh P , 3 Dental
Fatima K, Chaudhary G, Chaudhari PK -Evaluation of scientific quality of YouTube video content related to orthodontic pain management

( 4 )
Since search results frequently change at different moments, two new playlists were created to add videos for each search term in the YouTube™ library, and further both were merged to create a new playlist.The exclusion criteria used were the following: (1) Duplicate videos; (2) Videos not related to orthodontic pain management; (3) Videos longer than 20 minutes; Singh P, Fatima K, Chaudhary G, Chaudhari PK -Evaluation of scientific quality of youtube video content related to orthodontic pain management Dental Press J Orthod.2023;28(5):e232386 Videos in languages other than English; (5) Videos uploaded for entertainment purposes; (6) Videos uploaded for the purpose of advertising a product (Fig. 1).

(
Fig 1).The videos were assessed and evaluated independently by two different investigators (PS and KF), with 6 years and 5 years of experience respectively.The video assessment and

Singh P ,
Fatima K, Chaudhary G, Chaudhari PK -Evaluation of scientific quality of youtube video content related to orthodontic pain management Dental Press J Orthod.2023;28(5):e232386 uncertainty started?Generally poor quality and poor flow, some information listed but many important topics missing, of very limited use to patients Moderate quality, suboptimal flow, some important information is adequately discussed but others poorly discussed, somewhat useful for patients Good quality and generally good flow, most of the relevant information is listed, but some topics are not covered, useful for patients Excellent quality and excellent flow, very useful for patients

related to orthodontic pain management 12 Dental
Data were independently collected by the two investigators using a Microsoft Excel (Microsoft) spreadsheet, and statistical evaluations were performed using Statistical Package for the Social Sciences software (SPSS version 25; IBM Corp, SPSS Inc, Armonk, USA).Normality of the data distribution was evaluated using the Shapiro-Wilk test.The variables were expressed as mean ± standard deviation, numbers, and percentages.Chi-square tests were used for the comparison of the videos between Doctors and Non-doctors.The degree of agreement between investigators was assessed using Cohen's Kappa coefficient (κ) score, and Intra-class correlation coefficients (ICCs) were calculated to define intra-rater reliability.Pearson correlation coefficients were calculated to evaluate possible correlations between GQS, AVQ, and RS.Statistical significance was evaluated at p < 0.05.Singh P, Fatima K, Chaudhary G, Chaudhari PK -Evaluation of scientific quality of YouTube video content

Singh P ,
Fatima K, Chaudhary G, Chaudhari PK -Evaluation of scientific quality of youtube video content related to orthodontic pain management Dental Press J Orthod.2023;28(5):e232386

Figure 3 :
Figure 3: A) Percentages distribution of the Audio-visual Quality ( AVQ ) score between Doctors and Non-doctors videos.B) Percentages distribution of the Global Quality Score ( GQS ) scores between Doctors and Non-doctors videos .C) Percentages distribution of Reliability Score ( RS ) between Doctors and Non-doctors videos.
patients.The availability of good quality of information and content on YouTube™ is essential because the feedback on these videos affects the decision-making process of the patients regarding orthodontic treatment.The cooperation and knowledge of orthodontic patients have an integral role in successful treatment outcomes.One of the main issues that patients and parents always consider regarding treatment is the pain associated with it.The video characteristics of this study showed that YouTube™ users viewed videos about orthodontic treatment and the pain management related to it at high rates, often uploaded videos, and frequently interacted with other users via likes, dislikes, and comment features.The variety of content available on YouTube™ regarding orthodontic pain management included videos related to wire tightening, e-chains/elastics, separator placement, band placement, Invisalign, expansion appliances and functional appliances.
2he video viewing rate was calculated by dividing the total number of views by the number of days of the video on YouTube.2

Table 1 :
Video demographics for Doctor and Non-doctor uploaders.

Table 2 :
Quality of videos in Doctors and Non-doctors groups, expressed in percentage and absolute number.

Table 3 :
Correlation matrix displaying Pearson correlation coefficients amongst AVQ score, GQS, Reliability score, and video demographics.

Chaudhari PK -Evaluation of scientific quality of YouTube video content related to orthodontic pain management 18 Dental
Surprisingly, the Interaction rate was better in videos from the Doctors group.The reason behind this could be the lesser number of dislikes and lesser time elapsed since the upload of the Doctors videos.